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(54) Image processing apparatus for performing compression encoding in accordance with the 
viewpoint position 



(57) An image processing apparatus divides an 
image signal corresponding to one frame into a plurality 
of blocks, and performs compression encoding for each 
block. The apparatus includes a display unit for display- 
ing the image signal, a designation unit for designating 
a position desired by a user on the display unit, and a 
compression encoding unit for weighting the blocks and 
performing compression encoding therefor in accord- 
ance with the desired position designated by the desig- 
nation unit. 

An image processing apparatus divides an image 
signal corresponding to one frame into a plurality of 
blocks, the image signal being output from a camera 

FIG. 1 



unit having focus adjustment unit for performing focus 
adjustment, and performs compression encoding for 
each block. The apparatus includes a display unit for 
displaying the image signal, a designation unit for desig- 
nating a position desired by a user on the display unit, a 
compression encoding unit for weighting the blocks and 
performing compression encoding therefor in accord- 
ance with an output from the designation unit, and a 
control unit for controlling the focus adjustment unit in 
accordance with the desired position designated by the 
designation unit. 
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Description 

BACKGROUND OF THE INVENTION 

Field of the Invention 5 

The present invention relates to an image process- 
ing apparatus and, more particularly, to compression 
encoding processing for image signals. 

10 

Related Background Art 

As an apparatus for recording and reproducing 
image signals, a VTR based on a high-efficiency encod- 
ing method has been proposed. In this apparatus, the is 
pixels of a frame are digitized and grouped into blocks 
each consisting of a predetermined number of pixels; 
and entropy coding is then performed by orthogonal 
transformation (e.g., discrete cosine transform (DCT)) 
and quantization. 20 

As one of such conventional techniques, a home 
digital VTR format has been proposed. In this tech- 
nique, an existing television signal is digitized, and the 
resultant information is recorded after it is compressed 
to about 1/6 25 

Fig. 17 is a block diagram showing the arrangement 
of a VTR with a built-in camera as one of the proposed 
home digital VTRs. 

Referring to Fig. 17, the VTR includes a lens group 
1 , an image pickup element 2 such as a CCD, a camera 30 
signal processing circuit 3 for converting a signal from 
the image pickup element 2 into a television signal, an 
LCD display circuit 4 for displaying a photographed 
image on an LCD (liquid crystal display) monitor, an 
LCD monitor 5 used as a finder, and a block dividing 35 
and shuffling circuit 6 for dividing a television signal cor- 
responding to one frame into a plurality of blocks, and 
shuffling the blocks. 

A motion detection circuit 7 detects whether a pic-< 
ture exhibits a larger or small motion (the correlation 40 
between frames is larger or small). A DCT calculation 
weighting circuit 8 performs a weighting operation in 
accordance with DCT calculation and a spatial fre- 
quency component. A sort circuit 9 sorts DCT coeffi- 
cients, which are obtained by the DCT calculation 45 
weighting circuit 8 upon orthogonal transformation, in 
the order of increasing frequencies. 

A code amount estimation circuit 10 obtains a 
quantization step width . with which a code amount 
obtained by quantization of data of a predetermined so 
number of blocks and variable length encoding is made 
constant An adaptive quantization circuit 1 1 performs 
quantization upon reception of the estimation result 
from the code amount estimation circuit 10. A variable 
length encoding circuit 12 performs variable length 55 
encoding by using two-dimensional Huffman codes. A 
deshuffling circuit 13 des":uffles data to their original 
positions on a frame obtained before data compression. 
A recording processing circuit 14 adds signals required 



for a recording operation, e.g., an error correct!.'/) code, 
sync signals, and a pilot signal. The apparatjs also 
includes a magnetic head 15 and a magnetic tape 16. 

The operation of the digital VTR with the built-in 
camera having the above arrangement will be described 
below. 

First of all, an object image is formed on the image 
pickup element 2 such as a CCD through the lens group 
1 , and converted into an electrical signal. 

The electrical signal from the image pickup element 
2 is subjected to y correction, color matrix processing, 
and the like in the camera signal processing circuit 3 to 
form a television signal. 

A general VTR with a built-in camera (camcorder) 
allows an operator to monitor a photographed image 
through the finder. An output from the camera signal 
processing circuit 3 is converted into an LCD driving sig- 
nal by the LCD display circuit 4 to display an image on 
the LCD monitor 5. 

The output from the camera signal processing cir- 
cuit 3 is also sent to a recording system. The block 
dividing and shuffling circuit 6 divides and shuffles 
blocks. 

The output from the camera signal processing cir- 
cuit 3 may be analog or digital data. If the output is ana- 
log data, an A/D converter is to be connected to the 
input side of the block dividing and shuffling circuit 6. 

A method of dividing and shuffling blocks will be 
described below with reference to Figs. 2 and 3. 

First of all, signals corresponding to one frame, i.e., 
a luminance (Y) signal and color difference (R-Y and B- 
Y) signals, are divided into DCT blocks each consisting 
of (8 x 8) pixels. 

Subsequently, as shown in Fig. 2, four Y signal 
blocks, one R-Y signal block, and one B-Y signal block, 
i.e., a total of six DCT blocks, which are located at the 
'same position on the frame are grouped into a macro 
block as a unit. 

As shown in Fig. 3, one frame is divided into five 
areas A to E, and each area is divided into 10 blocks 
each consisting of 27 macro blocks. 

That is, one frame is divided into five areas in the 
horizontal direction and 10 areas in the vertical direc- 
tion. Each of these divided blocks is called a super- 
block. 

Shuffling is performed in units of macro blocks. In 
the subsequent circuits, one macro block is selected 
from each of the five areas A to E according to a prede- 
termined rule, and processing is performed such that 
the image data of the five macro blocks collected from 
the respective areas becomes fixed length data after 
data compression. 

The DCT calculation weighting circuit 8 performs 
DCT calculation in units of DCT blocks, and performs a 
weighting operation such that a component having a low 
spatial frequency exhibits a smaller distortion than a 
component having a high spatial frequency. 

The DCT calculation weighting circuit 8 performs 
intrafield processing when a picture exhibits a large 
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motion, and performs intraframe processing when a pic- 
ture exhibits a large motion. The motion detection circuit 
7 determines processing to be performed by the DCT 
calculation weighting circuit 8. 

The sort circuit 9 sorts DCT coefficients, which are s 
obtained by the DCT calculation weighting circuit 8 
upon orthogonal transformation, in the order of increas- 
ing frequencies. A quantization step width for the sorted 
image data is determined in the adaptive quantization 
circuit 11 in accordance with the frequency of an AC io 
component. As descried above, control is performed 
such that the data of the five macro blocks becomes 
fixed length data after compression. This control is per- 
formed by the code amount estimation circuit 10. The 
code amount estimation circuit 10 sets a parameter in 15 
accordance with the characteristics of an image corre- 
sponding to each DCT block, and obtains an quantiza- 
tion step width corresponding to the parameter and a 
frequency component, thereby realizing fixed length 
conversion with efficient bit allocation to each DCT 20 
block. 

Although the image data quantized by the adaptive 
quantization circuit 1 1 is encoded into variable length 
codes by the variable length encoding circuit 12, thefive 
macro blocks are almost kept to be fixed length data. 25 

The deshuff ling circuit 1 3 deshuff ies the image data 
compressed by the deshuff ling circuit 13 to the original 
positions on the frame. By deshuffling the data to the 
original positions on the frame, a picture in the fast play- 
back mode is made more legible. so 

The recording processing circuit 14 adds signals 
required for a recording operation, e.g., an error correc- 
tion code, sync signals, and a pilot signal, to the deshuf- 
fled image data. The resultant data is recorded on the 
magnetic tape 1 6 by the magnetic head 15. 35 

The above prior art is very effective when the entire 
fame of a picture is uniformly compressed. In general, 
however, a photographer tends to fix his/her eye on only 
an object to be photographed in a frame. In addition, 
when there are a plurality of objects at different dis- 40 
tances, since only one target object is in focus, it is 
highly possible that the remaining objects are out of 
focus. 

In spite of such a situation, however, according to 
the above prior art, since bits are uniformly allocated to 45 
the entire frame, many bits are used for even unneces- 
sary portions. 

SUMMARY OF THE INVENTION 

50 

The present invention has been made in considera- 
tion of the above situation, and has as a concern to pro- 
vide an image processing apparatus which can perform 
compression encoding with respect to the image data of 
a portion desired by a user while preventing a deteriora- ss 
tion in image quality. 

According to an aspect of the present invention, 
there is provided an image processing apparatus for 
dividing an image signal corresponding to one frame 



into a plurality of blocks, and performing compression 
encoding for each block, comprising display means for 
displaying the image signal, designation means for des- 
ignating a position desired by a user on the display 
means, and compression encoding means for weighting 
the blocks and performing compression encoding there- 
for in accordance with an output from the designation 
means. 

It is another concern of the present invention to pro- 
vide an image processing apparatus which allow focus 
adjustment of a camera unit with respect to the image 
data of a portion (an object to be photographed) desired 
by a user, and can perform compression encoding with 
respect to the image data while preventing a deteriora- 
tion in image quality. 

. According to another aspect of the present inven- 
tion, there is provided an image processing apparatus 
for dividing an image signal corresponding to one frame 
into a plurality of blocks, and performing compression 
encoding for each block, the image signal being output 
from a camera unit having focus adjustment means for 
performing focus adjustment, comprising display means 
for displaying the image signal, designation means for 
designating a position desired by a user on the display 
means, compression encoding means for weighting the 
blocks and performing compression encoding therefor 
in accordance with an output from the designation 
means, and control means for controlling the focus 
adjustment means in accordance with the desired posi- 
tion designated by the designation means. 

Other features and advantages of the invention will 
become apparent from the following detailed descrip- 
tion taken in conjunction with the accompanying draw- 
ings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a block diagram showing the arrangement 

of a VTR with a built-in camera according to the first 

embodiment of the present invention; 

Fig. 2 is a view for explaining macro blocks; 

Fig. 3 is a view for explaining shuffling; 

Fig. 4 is a view showing an image displayed on an 

LCD monitor 5; 

Fig. 5 is a view for explaining shuffling and a priority 
area; 

Figs. 6A to 6C are views for explaining changes in 
bit allocation to image data when a viewpoint detec- 
tion result is used; 

Fig. 7 is a block diagram showing the arrangement 
of a VTR with a built-in camera according to the 
second embodiment of the present invention; 
Fig. 8 is a view showing the relationship between 
the viewpoint of a photographer and an AF area; 
Fig. 9 is a block diagram showing the arrangement 
of a VTR with a built-in camera according to the 
third embodiment of the present invention; 
Fig. 10 is a view showing the relationship between 
the viewpoint of a photographer and an AF area; 
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Fig. 1 1 is a block diagram showing the arrangement 
of a VTR with a built-in camera according to the 
fourth embodiment of the present invention; 
Fig. 12 is a view for explaining measurement of the 
distance between the coordinates of each macro 
block and a viewpoint position; 
Figs. 13A to 13C are views showing changes in bit 
allocation to image data in the fourth embodiment 
when a viewpoint detection result is used; 
Fig. 1 4 is a block diagram showing the arrangement 
of a VTR with a built-in camera according to the fifth 
embodiment of the present invention; 
Figs. 15A and 15B are views showing the principle 
of a viewpoint detection method; 
Figs. 16A and 16B are views for explaining the out- 
put intensity of an image sensor 512 in Figs. 15A 
and 1 5B; and 

Fig. 17 is a block diagram showing the arrangement 
of a conventional VTR with a built-in camera which 
is used to record digital signals. 

DETAILED DESCRIPTION OF THE PREFERRED 
EMBODIMENTS 

Fig. 1 is a block diagram showing the arrangement 
of a VTR with a built-in camera according to the first 
embodiment of the present invention. The same refer- 
ence numerals in Fig. 1 denote the same parts as in Fig. 
1 1 , and a description thereof will be omitted. 

Referring to Fig. 1, an eyeball 101 of a photogra- 
pher monitors an image displayed on an LCD monitor 5. 

An infrared-emitting diode (IRED) 103 irradiates 
infrared light to detect the viewpoint position of the eye- 
ball 101. A CCD sensor 104 receives infrared light 
reflected by the eyeball 101. A viewpoint detection cir- 
cuit 102 drives the infrared-emitting diode 103, and ana- 
lyzes the viewpoint position on the basis of information 
from the CCD sensor 104. A viewpoint detection switch 

105 ON/OFF-controls the operation of the viewpoint 
detection circuit 102. A priority area generation circuit 

106 generates area information consisting of a coordi- 
nates group on a frame on the basis of viewpoint posi- 
tion information created by the viewpoint detection 
circuit 102. 

The operation of the VTR with the built-in camera 
having the arrangement in Fig. 1 will be described 
below. 

Similar to the prior art, first of all, an object image is 
formed on an image pickup element 2 through a lens 
group 1 and converted into an electrical signal. The sig- 
nal is then processed to display an image on the LCD 
monitor 5. 

If the viewpoint detection switch 105 is ON, the 
viewpoint detection circuit 102 drives the infrared-emit- 
ting diode 103 to irradiate infrared light onto the eyeball 
1 01 , while the eyeball 1 01 of the photographer monitors 
the image on the LCD monitor 5. The CCD sensor 104 
receives infrared light reflected by the eyeball 101. 
Viewpoint position information is then generated by 



using information from the CCD sensor 104. A view- 
point position detection method will be described later. 

The priority area generation circuit 106 generates 
area information consisting of a coordinates group on a 

5 frame on the basis of the viewpoint position information 
from the viewpoint detection circuit 102. Meanwhile, an 
image signal from a camera signal processing circuit 3 
is processed by a block dividing and shuffling circuit 6 t a 
motion detection circuit 7, a DCT calculation weighting 

10 circuit 8, and a sort circuit 9. and the resultant image 
data is supplied to a code amount estimation circuit 10' 
and an adaptive quantization circuit 1 1 , as in the prior 
art. 

The code amount estimation circuit 10' estimates a 
is code amount with a predetermined offset so that bit 
allocation is preferentially performed with respect to 
macro blocks included in a priority area generated by 
the priority area generation circuit 106 over the remain- 
ing macro blocks in a fixed length unit. With this opera- 
20 tion, the code amount estimation circuit 10' controls a 
quantization step width for the adaptive quantization cir- 
cuit 11. 

This operation will be described with reference to 
Figs. 5 and 6A to 6C. 

25 Fig. 4 shows an image on the LCD monitor 5. As 
shown in Fig. 4. a house is located in the center of the 
frame, and a tree is on the left side of the house. 

The photographer fixes his eye on an upper portion 
(position indicated by "X") of the tree on the left side of 

30 the house. If the viewpoint detection switch 105 is ON, 
the coordinates of the viewpoint on the frame are 
obtained by the viewpoint detection circuit 102. As a 
result, viewpoint position information is generated. The 
priority area generation circuit 106 generates priority 

35 area information consisting of a coordinates group indi- 
cated by the hatched portion in Fig. 5 on the basis of the 
viewpoint position information. In a fixed length unit, five 
macro blocks are collected from five areas A to E 
according to a predetermined rule. Referring to Fig. 5, a 

40 macro block MB A is located in the priority area. There- 
fore, bit allocation is preferentially performed with 
respect to this block over the remaining macro blocks. 

Figs. 6A to 6C are views showing changes in bit 
allocation upon operation of the viewpoint detection 

45 switch 105. 

Fig. 6A shows the data of five macro blocks consti- 
tuting one fixed length unit before compression. 

Fig. 6B shows bit allocation after data compression 
when the viewpoint detection switch 105 is OFF. 

so In this case, data D c , D B , D D , D A , and D E are 
obtained after conversion to variable length codes. The 
five macro blocks are almost fixed length codes. 

H this image is identical to that in the prior art, bit 
allocation is performed in the same manner. 

55 Fig. 6C shows bit allocation after data compression 
when the viewpoint detection switch 105 is ON. Since 
the macro block MB A is in the priority area, bit allocation 
is preferentially performed with respect to this macro 
block over the remaining macro blocks. Methods of 
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weighting a macro block in a priority area and the 
remaining macro blocks include a method of adding an 
offset to the image quality parameter of each DCT block 
to increase the amount of bits to be allocated, and a 
method of setting a predetermined difference or ratio 5 
between the quantization parameter of a macro block in 
a priority area and that of each of the remaining macro 
blocks. In any case, a bit amount D A of the macro block 
in the priority area after compression is larger than that 
when the viewpoint detection switch 1 05 is OFF. 10 

Generally, the viewpoint position of the eyeball 101 
slightly varies. However, the motions of the eyeball 101 
are averaged by the viewpoint detection circuit 102 or 
the priority area generation circuit 106. In addition, tim- 
ing adjustment with respect to an output from the sort 15 
circuit 9 is performed. 

Although the data quantized by the adaptive quan- 
tization circuit 1 1 is encoded into variable length codes 
by a variable length encoding circuit, the data of the five 
blocks are almost fixed length codes. 20 

A deshuffling circuit 13 deshuffles the compressed 
data to the original positions on the frame. A recording 
processing circuit 14 adds signals required for a record- 
ing operation, e.g., an error correction code, sync sig- 
nals, and a pilot signal, to the deshuffled image data. 25 
The resultant data is recorded on a magnetic tape 1 6 by 
a magnetic head 15. 

As described above, this embodiment exemplifies 
the VTR with the built-in camera. However, a signal from 
a circuit other than the camera signal processing circuit 30 
3 may be used. In addition, the present invention can be 
applied to an apparatus other than the recording appa- 
ratus. For example, a data transmission apparatus can 
also perform compression encoding processing for the 
image data of a portion desired by a user while prevent- 35 
ing a deterioration in image quality. 

Fig. 7 is a block diagram showing the arrangement 
of a VTR with a built-in camera according to the second 
embodiment of the present invention. 

The same reference numerals in Fig. 7 denote the to 
same parts as in Fig, 1 , and a description thereof will be 
omitted. 

Referring to Fig. 7, an AF area generation circuit 
201 generates an automatic focusing area on a frame 
on the basis of viewpoint position information generated 45 
by a viewpoint detection circuit 102. An AF processing 
circuit 202 generates a focus control signal by using a 
portion, of the high-frequency component of an image 
signal from a camera signal processing circuit 3, which 
is included in the automatic focusing area generated by so 
the AF area generation circuit 201 . A lens driving circuit 
203 drives a lens group 1 in accordance with the focus 
control signal generated by the AF processing circuit 
202. 

The operation of the VTR with the built-in camera 55 
having the arrangement in Fig. 7 will be described 
below. Assume that a house is located in the center of a 
frame, and a tree is on the left side of the house, and 
these objects are at different distances from a photogra- 



pher. Similar to the first embodiment, first of all, each 
object image is formed on an image pickup element 2 
through the lens group 1 , and converted into an electri- 
cal signal. The signal is the processed to display an 
image on an LCD monitor 5. 

If a viewpoint detection switch 105 is ON, the view- 
point detection circuit 102 drives an infrared-emitting 
diode 103 to irradiate infrared light onto an eyeball 101 
of the photographer, while the eyeball 101 monitors the 
image on the LCD monitor 5. A CCD sensor 104 
receives infrared light reflected by the eyeball 101, and 
generates viewpoint position information by using infor- 
mation from the CCD sensor 104. 

If the photographer fixes his/her eye on an upper 
portion of the tree, as shown in Fig. 8, viewpoint position 
information is generated from the viewpoint (position 
indicated by "X"). 

The AF area generation circuit 201 generates an 
automatic focusing area f AF AREA" in Fig. 8) on the 
basis of the viewpoint position information generated by 
the viewpoint detection circuit 102. The AF processing 
circuit 202 generates a focus control signal from the 
high-frequency component of the image signalfrom the 
camera signal processing circuit 3 and the automatic 
focusing area information. The lens driving circuit 203 
then drives the lens group 1 to focus the camera on the 
upper portion ("AF AREA" in Fig. 8) of the tree. At this 
time, the house in the center of the frame is out of focus. 
Meanwhile, a priority area generation circuit 106 gener- 
ates priority area information consisting of a coordi- 
nates group on the frame on the basis of the viewpoint 
position information from the viewpoint detection circuit 
102. 

Similar to the first embodiment, the image signal 
from the camera signal processing circuit 3 is proc- 
essed by a block dividing and shuffling circuity, a 
motion detection circuit 7, a DCT calculation weighting 
circuit 8, and a sort circuit 9. The resultant image data is 
supplied to a code amount estimation circuit 10' and an 
adaptive quantization circuit 1 1 . 

The code amount estimation circuit 10/ estimates a 
code amount with a predetermined offset such that bit 
allocation is preferentially performed with respect to a 
macro block included in a priority area generated by the 
priority area generation circuit 106 over the remaining 
macro blocks, thereby controlling a quantization step 
width for the adaptive quantization circuit 1 1 . 

That is, bit allocation is preferentially performed 
with respect to macro blocks near the upper portion of 
the tree in Fig. 8 over the macro blocks corresponding to 
the out-of-focus house in the center of the frame. 

As described above, in this embodiment, the view- 
point detection switch 105, the viewpoint detection cir- 
cuit 102, the infrared-emitting diode 103, and the CCD 
sensor 104 are used for both the automatic focusing 
function and the bit allocation function. However, these 
components may be separately used for these func- 
tions. In addition, one circuit may serve as both the AF 
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area generation circuit 201 and the priority area gener- 
ation circuit 106. 

The present invention is also effective for appara- 
tuses other than the recording apparatus, e.g., a data 
transmission apparatus. 

Fig. 9 is a block diagram showing the arrangement 
of a VTR with a built-in camera according to the third 
embodiment of the present invention. 

The same reference numerals in Fig. 9 denote the 
same parts as in Fig. 7, and a description thereof will be 
omitted. Only a method of designating a priority area in 
the third embodiment, which is different from that in the 
second embodiment, will be described below. Since 
other operations are the same as those in the second 
embodiment, a description thereof will be omitted. 

A direction switch 301 is used by a photographer to 
designate coordinates on a finder frame. A coordinates 
generation circuit 302 generates coordinates informa- 
tion in accordance with a designation from the direction 
switch 301. A coordinates designation switch 303 per- 
forms ON/OFF-controls the operation of the coordinates 
generation circuit 302. The operation of the VTR with 
the built-in camera having the arrangement in Fig. 9 will 
be described below. Assume that a house is located in 
the center of a frame, and a tree is on the left side of the 
house, and these objects are at different distances from 
a photographer. Similar to the first embodiment, first of 
all, each object image is formed on an image pickup ele- 
ment 2 through the lens group 1 , and converted into an 
electrical signal. The signal is the processed to display 
an image on an LCD monitor 5, 

If the coordinates designation switch 303 is ON, the 
coordinates generation circuit 302 generates coordi- 
nates information in accordance with a designation from 
the direction switch 301 . 

When the photographer wants to fix his/her eye on 
an upper portion (position indicated by "X") of the tree 
as shown in Fig. 10, he/she turns on the coordinates 
designation switch 303, and operates the direction 
switch 301 to bring the AF area in the frame near the 
position indicated by "X". 

An AF area generation circuit 201 generates an 
automatic focusing area (the AF area in Fig. 10) on the 
basis of the coordinates information generated by the 
coordinates generation circuit 302. An AF processing 
circuit 202 generates a focus control signal on the basis 
of the high-frequency component of an image signal 
from a camera signal processing circuit 3 and the auto- 
matic focusing area information, and a lens driving cir- 
cuit 203 drives the lens group 1 to focus the camera on 
the upper portion (the AF area in Fig. 10) of the tree. At 
this time, the house in the center is out of focus. A prior- 
ity area generation circuit 106 generates priority area 
information consisting of a coordinates group on the 
frame on the basis of the coordinates information from 
the coordinates generation circuit 302. 

Similar to the second embodiment, the image sig- 
nal from the camera signal processing circuit 3 is proc- 
essed by a block dividing and shuffling circuit 6, a 



motion detection circuit 7, a DCT calculation weighting 
circuit 8, and a sort circuit 9. The resultant image data is 
supplied to a code amount estimation circuit 10' and an 
adaptive quantization circuit 1 1 . 
5 The code amount estimation circuit 10' estimates a 
code amount with a predetermined offset such that bit 
allocation is preferentially performed with respect to 
macro blocks included in the priority area generated by 
the priority area generation circuit 106 over the remain- 
to ing macro blocks, thereby controlling a quantization 
step width for the adaptive quantization circuit 1 1 . 

That is, bit allocation is preferentially performed 
with respect to the macro blocks near the upper portion 
ol the tree in Fig. 10 over the macro blocks correspond- 
15 ing to the out-of-fbcus house in the center of the frame. 
Fig. 1 1 is a block diagram showing the arrangement 
of a VTR with a built-in camera according to the fourth 
embodiment of the present invention. The same refer- 
ence numerals in Fig. 1 1 denote the same parts as in 
20 Fig. 7, and a description thereof will be omitted. Only 
weighting processing to be performed in data compres- 
sion in the fourth embodiment, which is different from 
that in the second embodiment, will be described below. 
Other operations are the same as those in the second 
25 embodiment, and hence a description thereof will be 
omitted. 

Referring to Fig. 1 1 . a distance measurement cir- 
cuit 401 obtains the distance between the coordinates 
of a viewpoint position detected by a viewpoint detection 

30 circuit 102 and the coordinates of a macro block on a 
frame. A weighting parameter circuit 402 generates 
parameters for weighting five fixed length macro blocks. 

Assume that an image like the one shown in Fig. 4 
is displayed on an LCD monitor 5, and a photographer 

35 fixes his/her eye on an upper portion (position indicated 
by "X") of a tree on the left side. 

If a viewpoint detection switch 105 is ON, the view- 
point detection circuit 102 obtains coordinates indicat- 
ing a viewpoint. As shown in Fig. 12, the distance 

40 measurement circuit 401 receives the addresses of five 
fixed length macro blocks (MG A , MB B , MB C> MB D , and 
MBe), converts the addresses into coordinates, and 
obtains the distances (L A , L B , Lo k> and Le) between 
the coordinates of the viewpoint and those of the macro 

45 blocks. In this case, L A < Lq < Lq < Iq < L E , and the 
weighting parameter circuit 402 generates weighting 
parameters such that bit allocation is preferentially per- 
formed with respect to the macro blocks located nearer 
to the viewpoint. In this case, weighting parameters are 

so generated such that bit allocation is preferentially per- 
formed with respect to the macro block MB A nearest to 
the viewpoint, and the smallest number of bits are allo- 
cated to the macro block MBG E farthest from the view- 
point. 

55 A code amount estimation circuit 10' estimates a 
code amount on the basis of the data of the five macro 
blocks (MB C , MB B , MB D , MB A , and MB E ) from a sort cir- 
cuit 9, the image quality parameters of the respective 
DCT blocks, and the weighting parameters, and con- 
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trols a quantization step width for an adaptive quantiza- 
tion circuit 11.. 

Figs. 13A to 13C are views showing variations in bit 
allocation in accordance with the operation of the view- 
point detection switch 105. s 

Fig. 1 3 A shows the data of five macro blocks as one 
fixed length unit before compression. 

Fig. 13B shows bit allocation after data compres- 
sion when the viewpoint detection switch 105 is OFF. 

In this case, data D c . D B , D D , D A , and D E are 10 
obtained after conversion to variable length codes. The 
five macro blocks are almost fixed length codes. If this 
image is identical to that in the prior art, bit allocation is 
performed in the same manner. 

Fig. 13C shows bit allocation after data compres- is 
sion when the viewpoint detection switch 105 is ON. 

Bit allocation is preferentially performed with 
respect to the macro block MB A , but the number of bits 
allocated to the macro block MB E is small. Methods of 
weighting macro blocks include a method of adding an 20 
offset to the image quality parameter of each DCT block 
to increase/decrease the number of bits to be allocated, 
a method of giving a slope to the quantization parame- 
ter of each macro block, and the like. In any case, a bit 
amount D A of the macro block MB A nearest to the view- 25 
point position after compression is larger than that when 
the viewpoint detection switch 105 is OFF, but a bit 
amount D E of the macro block MB E farthest from the 
viewpoint position is smaller than that when the view- 
point detection switch 1 05 is OFF so 

Processing other than that described above is the 
same as that in the second embodiment, and hence a 
description thereof will be omitted. 

Fig. 14 is a block diagram showing the arrangement 
of a VTR with a built-in camera according to the fifth 35 
embodiment of the present invention. The same refer- 
ence numerals in Fig. 14 denote the same parts as in 
Figs. 9 or 11, and a description thereof will be omitted. 
Only a method of designating a priority area in the fifth 
embodiment, which is different from that in the fourth 40 
embodiment, will be described below. Since other oper- 
ations are the same as those in the fourth embodiment, 
a description thereof will be omitted. 

The operation of the VTR with the built-in camera 
having the arrangement in Fig. 14 will be described 45 
below. Assume that a house is located in the center of a 
frame, and a tree is on the left side of the house, and 
these objects are at different distances from a photogra- 
pher. Similar to the fourth embodiment, first of all, each 
object image is formed on an image pickup element 2 so 
through the lens group 1 , and converted into an electri- 
cal signal. The signal is the processed to display an 
image on an LCD monitor 5. 

If a coordinates designation switch 303 is ON. a 
coordinates generation switch 302 generates coordi- ss 
nates information in accordance with a designation from 
a direction switch 301. 

When the photographer wants to fix his/her eye on 
an upper portion (position indicated by "X") of the tree 



as shown in Fig. 10, he/she turns on the coordinates 
designation switch 303, and operates the direction 
switch 301 to bring the AF area in the frame near the 
position indicated by "X\ 

An AF area generation circuit 201 generates an 
automatic focusing area (the AF area in Fig. 10) on the 
basis of the coordinates information generated by the 
coordinates generation circuit 302. An AF processing 
circuit 202 generates a focus control signal on the basis 
of the high-frequency component of an image signal 
from a camera signal processing circuit 3 and the auto- 
matic focusing area information, and a lens driving cir- 
cuit 203 drives the lens group 1 to focus the camera on 
the upper portion (the AF area in Fig. 10) of the tree. At 
this time, the house in the center of the frame is out of 
focus. Meanwhile, a distance measurement circuit 401 
and a weighting parameter circuit 402 perform process- 
ing similar to that in the fourth embodiment on the basis 
of the coordinates information from the coordinates 
generation switch 302. 

In addition, an image signal from the camera signal 
processing circuit 3 is processed in the same manner as 
in the fourth embodiment, and the resultant data is 
recorded on a magnetic tape 16. 

Tlie principle of viewpoint detection, i.e., detection 
of the viewpoint of a photographer on a finder used in 
the first, second, and fourth embodiments will be 
described below. 

Figs. 15A and 15B show the principle of a viewpoint 
detection method. Fig. 15A shows the upper surface of 
the finder. Fig. 1 5B shows a side surface of the finder. 

Referring to Figs. 15A and 15B, light sources 506a 
and 506b such as light-emitting diodes (IREDs).emit 
infrared light to which the photographer is insensitive. 
The light sources 506a and 506b are arranged to be 
symmetrical (see Fig. 15A) about the optical axis of an 
imaging lens 51 1 in the x direction (horizontal direction) 
and slightly below (see Fig. 15B) the optical axis in the 
y direction (vertical direction). These light sources illu- 
minate an eyeball of the photographer with divergence 
light. 

Part of illumination light reflected by the eyeball is 
focused onto an image sensor 512 through the imaging 
lens 511. 

Referring to Figs. 15A and 15B, the light-emitting 
diodes 506a and 506b correspond to the ILED 103 in 
Figs. 1. 7, and 11, and the image sensor 512 corre- 
sponds to the CCD sensor 104 in Figs. 7 and 1 1 . 

Figs. 16A and 16B are views for explaining the out- 
put intensity of the image sensor 512. Note that the 
same reference numerals in Figs. 16A and 16B denote 
the same parts as in Figs. 15A and 15B. 

Fig. 16A is a schematic view showing an eyeball 
image projected on the image sensor 512. Fig. 16B is a 
graph showing the output intensity of the image sensor 
512. 

The viewpoint detection method will be described 
below with reference to Figs. 15A, 15b, 16A, and 16B, 
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Consider a horizontal plane first. Referring to Fig. 
15A, infrared light emitted from the light source 506b 
illuminates a cornea 510 of an eyeball 508 of the pho- 
tographer. A cornea reflection image d (virtual image) r 
formed by infrared light reflected by the surface of the 
cornea 510 is focused by the imaging lens 511. The 
resultant image is formed at a position d' on the image 
sensor 512. 

Similarly, infrared light emitted from the light source 
506a illuminates the cornea 510. 

In this case, cornea reflection image e (virtual 
image) formed by infrared light reflected by the surface 
of the cornea 510 is focused by the imaging lens 51 1. 
The resultant image is formed at a position e' on the 
image sensor 512. 

Light beams from end portions a and b of an iris 
504 are formed into images of the end portions a and b 
at posroons a' and b' on the image sensor 512 through 
the imaging lens 51 1. Assume that a rotational angle G 
of the optical axis of the eyeball 508 with respect to the 
optical axis of the imaging lens 511 is small. In this 
case, if the x-coordi nates of the end portions a and b of 
the iris 504 are represented by xa and xb, respectively, 
many points ("X" in Fig. 16A) can be obtained as the 
coordinates xa and xb on the image sensor. 

A pupil center xc is calculated by the least squares 
method. Letting xo be the x-coordinate of a curvature 
center o of the cornea 510, a rotational angle 6x with 
respect to the optical axis of the eyeball 508 is given by 

oc x sinOx = xc - xo (1) 

In consideration of a predetermined correction value 8x 
at a middle point k between the cornea reflection 
images d and e, the x-coordinate xo is given as follows: 

xk = (xd + xe)/2 (2) 

xo = (xd + xe)/2 + i>x 

In this case, the correction value Sx is a numerical value 
geometrically obtained from a method of mounting the 
apparatus, the eyeball distance, and the like. A descrip- 
tion of a method of calculating this value will be omitted. 
With a substitution of equation (1) into equation (2), the 
rotational angle Ox is given by 

ex = arcsin[[xc - {(xd + xe)/2 + 8x}]/oc] (3) 

In addition, equation (3) can be written into equation (4) 
as follows, with a prime "*" being added to the coordi- 
nates of the respective feature points projected on the 
image sensor: 

Ox = arcsin[[xc' - {(xd 1 + xe')/2 + 8x'}]/oc/M * (4) 

where p is the magnification determined by a distance 
sze between the imaging lens 51 1 and the eye. In prac- 



tice, the magnification (3 is obtained as a function of a 
distance |xd' - xe'| between cornea reflection images. 

Consider a vertical plane next. Fig. 15B shows a 
corresponding arrangement. In this case, cornea reflec- 

5 tion images formed by the two IREDs 506a and 506b 
are focused at the same position i. 

A method of calculating a rotational angle Gy of the 
eyeball is almost the same as that in the case of the hor- 
izontal plane, but only equation (2) is different. Letting 

10 yo be the y-coordinate of the curvature center o of the 
cornea, 

yo = yi + Sy (5) 

15 where Sy is a numerical value geometrically obtained 
from a method of mounting the apparatus, the eyeball 
distance, and the like. A description of a method of cal- 
culating this value will be omitted. The rotational angle 
Gy in the vertical direction is given by 

20 

Gy = arcsinflyc' - (yi* + 8y')]/oc/p] *(6) 

The coordinates (xn, yn) of a position on a finder 
frame of the video camera are obtained on the horizon- 
25 tal and vertical planes, respectively, as follows, using a 
constant m determined by a finder optical system: 

xn = m x arcsin[[xc' - {(xd 1 + xe')/2 + Sx}]/oc/p] (7) 

30 yn = m x arcsinf[yc' - (yi' + 5y')]/oc/p] (8) 

As is apparent from Fig. 16A, pupil edges are 
detected by using the leading edge (xb') and trailing 
edge (xa 1 ) of an image sensor output. In addition, the 

35 coordinate of a cornea reflection image are obtained by 
using steep leading edge portions (xe') and (xd*). 

Other embodiments and modifications of the inven- 
tion can be made without departing from the spirit and 
scope of the invention. 

40 In other words, the foregoing description of embod- 
iments has been given for illustrative purpose only and 
not to be construed as imposing any limitation in every 
respect. 

The scope ol the invention is, therefore, to be deter- 
45 mined solely by the following claims and not limited by 
the text of the specifications and alterations made within 
a scope equivalent to the scope of the claims fall within 
the true spirit and scope of the invention. 

so Claims 

1. An image processing apparatus for dividing an 
image signal corresponding to one frame into a plu- 
rality of blocks, and performing compression encod- 
55 ing for each block, comprising: 

a) display means for displaying the image sig- 
nal; 
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b) detection means for detecting a viewpoint 
position of an operator on said display means; 
and 

c) compression encoding means for weighting 
the blocks and performing compression encod- s 
ing therefor in accordance with the viewpoint 
position detected by said detection means. 

2. An image processing apparatus for dividing an 
image signal corresponding to one frame into a plu- w 
rality of blocks, and performing compression encod- 
ing for each block, the image signal being output 
from a camera unit having focus adjustment means 

for performing focus adjustment, comprising: 

15 

a) display means for displaying the image sig- 
nal; 

b) detection means for detecting a viewpoint 
position of an operator on said display means; 

c) compression encoding means for weighting 20 
the blocks and performing compression encod- 
ing therefor in accordance with the viewpoint 
position detected by said detection means; and 

d) control means for controlling said focus 
adjustment means in accordance with the 25 
viewpoint position detected by said detection 
means. 

3. An image processing apparatus for dividing an 
image signal corresponding to one frame into a plu- 30 
rality of blocks, and performing compression encod- 
ing for each block, comprising: 

a) display means for displaying the image sig- 
nal; 35 

b) designation means for designating a position 
desired by a user on said display means; and 

c) compression encoding means for weighting 
the blocks and performing compression encod- 
ing therefor in accordance with an output from 40 
said designation means. 

4. An image processing apparatus for dividing an 
image signal corresponding to one frame into a plu- 
rality of blocks, and performing compression encod- 45 
ing for each block, the image signal being output 
from a camera unit having focus adjustment means 

for performing focus adjustment, comprising: 

a) display means for displaying the image sig- so 
nal; 

b) designation means for designating a position 
desired by a user on said display means; 

c) compression encoding means for weighting 
the blocks and performing compression encod- 55 
ing therefor in accordance with an output from 
said designation means; and 

d) control means for controlling said focus 
adjustment means in accordance with the 



desired position designated by said designa- 
tion means. 

5. Apparatus according to any one of claims 1 to 4, 
wherein said compression encoding means detects 
distances between the desired position designated 
by said designation means and the blocks, and per- 
forms weighting processing for the blocks on the 
basis of the detection result. 

6. Apparatus according to any preceding claim, 
wherein said compression encoding means 
includes orthogonal transformation means for per- 
forming orthogonal transformation of the image sig- 
nal, and quantization means for quantizing the 
image signal having undergone orthogonal trans- 
formation. 

7. Apparatus according to claim 6, wherein said com- 
pression encoding means weights the blocks by 
controlling a quantization step of said quantization 
means with respect to the blocks in accordance 
with the desired position designated by said desig- 
nation means. 

8. Apparatus according to claim 7, wherein said 
orthogonal transformation means is discrete cosine 
transform. 

9. A camera including image pick-up means, means 
for detecting the new point of a user of the camera 
and means for carrying out compression encoding 
on the image signal generated by the pick-up 
means in a manner determined by the new point as 
detected by said detecting means. 
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